Skip to content

feat: implement a paginate in reverse flag#4

Closed
fusion44 wants to merge 6 commits intovincenzopalazzo:macros/rpc_filteringfrom
fusion44:macros/rpc_filtering
Closed

feat: implement a paginate in reverse flag#4
fusion44 wants to merge 6 commits intovincenzopalazzo:macros/rpc_filteringfrom
fusion44:macros/rpc_filtering

Conversation

@fusion44
Copy link
Copy Markdown

@fusion44 fusion44 commented Apr 1, 2023

Since we don't fetch the full database anymore, we need a way to paginate through the data in reverse order. I've implemented a flag to the paginator argument.

When I run the code it crashes at

lightning/db/bindings.c

Line 41 in 41aae43

assert(pos < tal_count(stmt->bindings));

Any idea why?

vincenzopalazzo and others added 6 commits March 23, 2023 11:14
@vincenzopalazzo
feat: introduce callback for paginator
e0e967c 
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
@vincenzopalazzo
draft implementing the core of the paginator
7eebbbb 
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
@vincenzopalazzo
tests: add the paginator test suite
bc2e33e 
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
@vincenzopalazzo
fix: rework the paginator API
7401722 
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
@vincenzopalazzo
feat: add an example of paginator with limit and offset
e3312f2 
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
@fusion44
feat: implement a paginate in reverse flag
41aae43
vincenzopalazzo pushed a commit that referenced this pull request Jun 7, 2023
@morehouse @cdecker
db: avoid calling memcpy on NULL
45cb2b1 
It is possible for db_column_bytes() to return 0 and for
db_column_blob() to return NULL even when db_column_is_null() returns
false. We need to short circuit in this case.

Detected by UBSan:

  db/bindings.c:479:12: runtime error: null pointer passed as argument 2, which is declared to never be null
  /usr/include/string.h:44:28: note: nonnull attribute specified here

  #0 0x95f117 in db_col_arr_ db/bindings.c:479:2
  #1 0x95ef85 in db_col_channel_type db/bindings.c:459:32
  #2 0x852c03 in wallet_stmt2channel wallet/wallet.c:1483:9
  #3 0x81f396 in wallet_channels_load_active wallet/wallet.c:1749:23
  #4 0x81f03d in wallet_init_channels wallet/wallet.c:1765:9
  #5 0x72f1f9 in load_channels_from_wallet lightningd/peer_control.c:2257:7
  #6 0x672856 in main lightningd/lightningd.c:1121:25
vincenzopalazzo pushed a commit that referenced this pull request Jun 7, 2023
@morehouse @cdecker
devtools: call common_setup in main
e81bf8b 
Fixes nullability errors detected by UBSan:

wire/fromwire.c:173:46: runtime error: null pointer passed as argument 1, which is declared to never be null
external/libwally-core/src/secp256k1/include/secp256k1.h:432:3: note: nonnull attribute specified here
    #0 0x65214a in fromwire_secp256k1_ecdsa_signature wire/fromwire.c:173:6
    #1 0x659500 in printwire_secp256k1_ecdsa_signature devtools/print_wire.c:331:1
    #2 0x646ba2 in printwire_channel_update wire/peer_printgen.c:1900:7
    #3 0x637182 in printpeer_wire_message wire/peer_printgen.c:128:11
    #4 0x65a097 in main devtools/decodemsg.c:85:10
@fusion44 fusion44 closed this by deleting the head repository Nov 18, 2025
vincenzopalazzo pushed a commit that referenced this pull request Jan 7, 2026
@rustyrussell
plugins: fix %*.s typo.
8f6d3d8 
And add a check for new uses creeping in, since it got cut & paste
everywhere.

This means "this is a valid string, but truncate it to this many characters"
vs "%.*s" which means "only read this many characters of string":

```
['lightningd-3 2025-10-23T02:31:40.890Z **BROKEN** plugin-funder: Plugin marked as important, shutting down lightningd!']
--------------------------- Captured stderr teardown ---------------------------
    #0 0x557da58ad1dc in printf_common(void*, char const*, __va_list_tag*) asan_interceptors.cpp.o
    #1 0x557da5aff814 in json_out_addv /home/runner/work/lightning/lightning/ccan/ccan/json_out/json_out.c:239:11
    #2 0x557da59740ce in plugin_logv /home/runner/work/lightning/lightning/plugins/libplugin.c:1777:2
    #3 0x557da5969b6f in plugin_log /home/runner/work/lightning/lightning/plugins/libplugin.c:1934:2
    #4 0x557da595c4f6 in datastore_del_success /home/runner/work/lightning/lightning/plugins/funder.c:161:2
    #5 0x557da598b837 in handle_rpc_reply /home/runner/work/lightning/lightning/plugins/libplugin.c:1072:10
    #6 0x557da598a4b0 in rpc_conn_read_response /home/runner/work/lightning/lightning/plugins/libplugin.c:1361:3
    #7 0x557da5adbea5 in next_plan /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:60:9
    #8 0x557da5ae06ff in do_plan /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:422:8
    ElementsProject#9 0x557da5adfb58 in io_ready /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:439:10
    ElementsProject#10 0x557da5aec2ce in io_loop /home/runner/work/lightning/lightning/ccan/ccan/io/poll.c:455:5
    ElementsProject#11 0x557da59757ac in plugin_main /home/runner/work/lightning/lightning/plugins/libplugin.c:2409:3
    ElementsProject#12 0x557da594fe23 in main /home/runner/work/lightning/lightning/plugins/funder.c:1723:2
    ElementsProject#13 0x7f6572229d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    ElementsProject#14 0x7f6572229e3f in __libc_start_main csu/../csu/libc-start.c:392:3
    ElementsProject#15 0x557da588b584 in _start (/home/runner/work/lightning/lightning/plugins/funder+0x10d584) (BuildId: 71ba63ab577fc6fa60573d3e8555f6db7d5c584d)

0x624000009d28 is located 0 bytes to the right of 7208-byte region [0x624000008100,0x624000009d28)
allocated by thread T0 here:
    #0 0x557da590e7f6 in __interceptor_realloc (/home/runner/work/lightning/lightning/plugins/funder+0x1907f6) (BuildId: 71ba63ab577fc6fa60573d3e8555f6db7d5c584d)
    #1 0x557da5b2149b in tal_resize_ /home/runner/work/lightning/lightning/ccan/ccan/tal/tal.c:755:13
    #2 0x557da59f2032 in membuf_tal_resize /home/runner/work/lightning/lightning/common/utils.c:203:2
    #3 0x557da5b03934 in membuf_prepare_space_ /home/runner/work/lightning/lightning/ccan/ccan/membuf/membuf.c:45:12
    #4 0x557da59d4289 in jsonrpc_io_read_ /home/runner/work/lightning/lightning/common/jsonrpc_io.c:127:2
    #5 0x557da598a635 in rpc_conn_read_response /home/runner/work/lightning/lightning/plugins/libplugin.c:1366:9
    #6 0x557da5adbea5 in next_plan /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:60:9
    #7 0x557da5ae06ff in do_plan /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:422:8
    #8 0x557da5adfb58 in io_ready /home/runner/work/lightning/lightning/ccan/ccan/io/io.c:439:10
    ElementsProject#9 0x557da5aec2ce in io_loop /home/runner/work/lightning/lightning/ccan/ccan/io/poll.c:455:5
    ElementsProject#10 0x557da59757ac in plugin_main /home/runner/work/lightning/lightning/plugins/libplugin.c:2409:3
    ElementsProject#11 0x557da594fe23 in main /home/runner/work/lightning/lightning/plugins/funder.c:1723:2
    ElementsProject#12 0x7f6572229d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

SUMMARY: AddressSanitizer: heap-buffer-overflow asan_interceptors.cpp.o in printf_common(void*, char const*, __va_list_tag*)
Shadow bytes around the buggy address:
  0x0c487fff9350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c487fff9360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c487fff9370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c487fff9380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c487fff9390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c487fff93a0: 00 00 00 00 00[fa]fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c487fff93f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==26122==ABORTING
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@fusion44 @vincenzopalazzo